Assessor Resource

MSL925002
Analyse measurements and estimate uncertainties

Assessment tool

Version 1.0
Issue Date: May 2024


This unit of competency covers the ability to use statistical analysis to estimate and report measurement uncertainty in accordance with the ISO Guide to the Expression of Uncertainty in Measurement (or its replacement Standard). Personnel are required to review their estimates of measurement uncertainty to assist with making decisions on the fitness for purpose of the measurements.

This unit of competency is applicable to laboratory personnel who work in calibration and testing facilities, process and interpret data and are required to determine uncertainties using standard methods. The rigour required in estimating uncertainty will depend on the required accuracy of the particular calibration, test or measurement.

While no specific licensing or certification requirements apply to this unit at the time of publication, laboratory operations are governed by relevant legislation, regulations and/or external accreditation requirements. Local requirements should be checked.

You may want to include more information here about the target group and the purpose of the assessments (eg formative, summative, recognition)



Evidence Required

List the assessment methods to be used and the context and resources required for assessment. Copy and paste the relevant sections from the evidence guide below and then re-write these in plain English.

Elements describe the essential outcomes.

Performance criteria describe the performance needed to demonstrate achievement of the element.

1

Identify the measured quantity and the uncertainty components

1.1

Specify an equation for the measurement

1.2

List uncertainty components that are associated with each input in the equation

2

Determine the size of each uncertainty component

2.1

Calculate the standard deviations and standard deviation of the mean from the measurement results

2.2

Use calibration reports, manufacturer's specifications, quality control and validation data, and experimental data to collect other available information on the uncertainty components

3

Reduce each uncertainty component to a standard uncertainty

3.1

Allocate an appropriate distribution for each uncertainty component

3.2

Calculate the standard uncertainties

4

Calculate an expanded uncertainty to the required confidence level

4.1

Calculate the sensitivity coefficient for each uncertainty component

4.2

Calculate a combined standard uncertainty

4.3

Determine an appropriate coverage factor based on the degrees of freedom associated with each uncertainty component

4.4

Calculate the expanded uncertainty

5

Report the expanded uncertainty

5.1

Report the result and uncertainty to an appropriate number of significant figures

5.2

Report the confidence level and coverage factor

5.3

Determine the appropriateness of the size of the expanded uncertainty relative to the tolerance or required accuracy of the test

5.4

Determine the fitness for purpose of the expanded uncertainty relative to the use of the measurement result

Evidence of competence in this unit must satisfy all of the requirements of the elements and performance criteria, and include demonstration of:

using scientific notation, correct units and correct number of significant figures

evaluating formulae containing powers, exponents and logarithms functions

preparing and interpreting linear graphs

using statistical analysis to estimate and report measurement uncertainty in accordance with the ISOGuide to the Expression of Uncertainty in Measurement

preparing a fully documented, realistic uncertainty budget that is appropriate for the application

calculating a combined standard uncertainty using root-sum-of-squares, accounting for correlations where necessary

gathering information about uncertainty components from calibration reports or reference material report

calculating sensitivity coefficients either experimentally or by partial differentiation

calculating expanded uncertainty

using spreadsheets to calculate uncertainties

deciding if the uncertainty is suitable for the accuracy required for the test and establishing whether it is fit for purpose using the tolerance to uncertainty ratio (TUR)

using and interpreting mean, standard deviation, standard deviation of the mean, degrees of freedom, histograms and frequency plots, probability, normal probability plots and control charts

using the student's t-table to get a coverage factor for a particular level of confidence

using and interpreting significance tests, such as t-test, f-test and analysis of variance (ANOVA), variances, standard deviation of prediction and linear regression

using regression methods for calibration, linearity checks and comparing analytical methods

using and interpreting normal, rectangular, triangular distributions and the factors used to reduce each to a standard uncertainty

allocating degrees of freedom to each uncertainty component using the Welch-Satterthwaite equation

reporting results and uncertainties in the required formats.

Must provide evidence that demonstrates knowledge of:

the steps in the measurement, test or calibration involved

the difference between errors, corrections and uncertainties

uncertainty in the uncertainty estimation process

uncertainty components that are common to the use of an instrument

uncertainty components that arise due to the instrument being used under different conditions to those when it was calibrated

uncertainty components, such as:

calibration uncertainty

instability or drift in the calibrated instrument

repeatability of the results

resolution or readability of the instrument

environmental influences, such as temperature, air pressure, humidity, vibration, electrical noise and gravity

reference material uncertainty

factors arising from using an instrument under a different operating environment or procedures (e.g. orientation of a transducer and immersion depth of a temperature probe)

reproducibility of quality control data

procedures for determining the uncertainty components associated with each of the inputs and whether they are significant and for applying appropriate corrections

manufacturer's specifications (e.g. instrument drift specification and reference materials)

procedures for determining uncertainty components from quality control data

the concept of degrees of freedom

the characteristics of a valid measurement

reporting requirements, such as the uncertainty in measurement (GUM), National Association of Testing Authorities (NATA) or other applicable reference material

confidence levels required by the National Measurement Act 1960 (e.g. 95% for most applications, but others may require a higher confidence level).

Judgement of competence must be based on holistic assessment of the evidence. Assessment methods must confirm consistency of performance over time, rather than a single assessment event.

This unit of competency is to be assessed in the workplace or a simulated workplace environment. A simulated workplace environment must reflect realistic operational workplace conditions that cover all aspects of workplace performance, including the environment, task skills, task management skills, contingency management skills and job role environment skills.

Foundation skills are integral to competent performance of the unit and should not be assessed separately.

Assessment processes and techniques must be appropriate to the language, literacy and numeracy requirements of the work being performed and the needs of the candidate.

Knowledge evidence may be collected concurrently with performance evidence or through an independent process, such as workbooks, written assessments or interviews (provided a record is kept in each case).

This unit of competency may be assessed with:

MSL904001 Perform standard calibrations

MSL905001 Perform non-standard calibrations

Holistic assessment methods include:

review of data worksheets, calculations, spreadsheets, databases, statistical analysis, graphs and/or tables prepared by the candidate

questions to assess understanding of relevant procedures, trends in data and sources of uncertainty

review of reports and records prepared by the candidate

feedback from supervisors and peers regarding the candidate's ability to estimate uncertainty in accordance with workplace procedures.

Access is required to all instruments, equipment, materials, workplace documentation, procedures and specifications associated with this unit, including, but not limited to:

data sets and records

test methods and description of test set-up

computer and relevant software or laboratory information system

workplace procedures.

Assessors must satisfy the assessor competency requirements that are in place at the time of the assessment as set by the VET regulator.

The assessor must demonstrate both technical competency and currency.

Technical competence can be demonstrated through:

relevant VET or other qualification/Statement of Attainment AND/OR

relevant workplace experience.

Currency can be demonstrated through:

performing the competency being assessed as part of current employment OR

having consulted with a laboratory about performing the competency being assessed within the last twelve months.


Submission Requirements

List each assessment task's title, type (eg project, observation/demonstration, essay, assingnment, checklist) and due date here

Assessment task 1: [title]      Due date:

(add new lines for each of the assessment tasks)


Assessment Tasks

Copy and paste from the following data to produce each assessment task. Write these in plain English and spell out how, when and where the task is to be carried out, under what conditions, and what resources are needed. Include guidelines about how well the candidate has to perform a task for it to be judged satisfactory.

Elements describe the essential outcomes.

Performance criteria describe the performance needed to demonstrate achievement of the element.

1

Identify the measured quantity and the uncertainty components

1.1

Specify an equation for the measurement

1.2

List uncertainty components that are associated with each input in the equation

2

Determine the size of each uncertainty component

2.1

Calculate the standard deviations and standard deviation of the mean from the measurement results

2.2

Use calibration reports, manufacturer's specifications, quality control and validation data, and experimental data to collect other available information on the uncertainty components

3

Reduce each uncertainty component to a standard uncertainty

3.1

Allocate an appropriate distribution for each uncertainty component

3.2

Calculate the standard uncertainties

4

Calculate an expanded uncertainty to the required confidence level

4.1

Calculate the sensitivity coefficient for each uncertainty component

4.2

Calculate a combined standard uncertainty

4.3

Determine an appropriate coverage factor based on the degrees of freedom associated with each uncertainty component

4.4

Calculate the expanded uncertainty

5

Report the expanded uncertainty

5.1

Report the result and uncertainty to an appropriate number of significant figures

5.2

Report the confidence level and coverage factor

5.3

Determine the appropriateness of the size of the expanded uncertainty relative to the tolerance or required accuracy of the test

5.4

Determine the fitness for purpose of the expanded uncertainty relative to the use of the measurement result

This field allows for different work environments and conditions that may affect performance. Essential operating conditions that may be present (depending on the work situation, needs of the candidate, accessibility of the item, and local industry and regional contexts) are included.

Standards, codes, procedures and/or workplace requirements

Standards, codes, procedures and/or enterprise requirements include the latest version of one or more of:

Australian and international standards and guides, such as:

competency requirements for testing and calibration laboratories, quality management and measurement management

ISO 5725 Accuracy (trueness and precision) of measurement methods and results

ISO/IEC Guide 98-3:2008 Uncertainty of Measurement - Part 3 Guide to the Expression of Uncertainty in Measurement

Eurachem/CITAC Guide CG4 Quantifying uncertainty in analytical measurement, Eurolab technical report and Nordtest

National Association of Testing Authorities (NATA) accreditation program requirements and NATA technical notes

national measurement regulations and guidelines

Australian codes, such as of good manufacturing practice for medicinal products (GMP), and principles of good laboratory practice (GLP)

workplace documents, such as quality manual, customer quality plan, equipment manuals and warranty, supplier catalogues and handbooks

validated sampling and test procedures

Data

Data includes:

worksheets, spreadsheets or databases linked to information management systems

the results of tests, measurements and analyses

Records

Records include information associated with one or more of:

purchase of equipment and materials and service records

manufacturer datasheets

calibration reports

history of calibration and test results

Work health and safety (OHS) and environmental management requirements

WHS and environmental management requirements include:

· complying with WHS and environmental management requirements at all times, which may be imposed through state/territory or federal legislation. These requirements must not be compromised at any time

· applying standard precautions relating to the potentially hazardous nature of samples

accessing and applying current industry understanding of infection control issued by the National Health and Medical Research Council (NHMRC) and State and Territory Departments of Health, where relevant

Evidence of competence in this unit must satisfy all of the requirements of the elements and performance criteria, and include demonstration of:

using scientific notation, correct units and correct number of significant figures

evaluating formulae containing powers, exponents and logarithms functions

preparing and interpreting linear graphs

using statistical analysis to estimate and report measurement uncertainty in accordance with the ISOGuide to the Expression of Uncertainty in Measurement

preparing a fully documented, realistic uncertainty budget that is appropriate for the application

calculating a combined standard uncertainty using root-sum-of-squares, accounting for correlations where necessary

gathering information about uncertainty components from calibration reports or reference material report

calculating sensitivity coefficients either experimentally or by partial differentiation

calculating expanded uncertainty

using spreadsheets to calculate uncertainties

deciding if the uncertainty is suitable for the accuracy required for the test and establishing whether it is fit for purpose using the tolerance to uncertainty ratio (TUR)

using and interpreting mean, standard deviation, standard deviation of the mean, degrees of freedom, histograms and frequency plots, probability, normal probability plots and control charts

using the student's t-table to get a coverage factor for a particular level of confidence

using and interpreting significance tests, such as t-test, f-test and analysis of variance (ANOVA), variances, standard deviation of prediction and linear regression

using regression methods for calibration, linearity checks and comparing analytical methods

using and interpreting normal, rectangular, triangular distributions and the factors used to reduce each to a standard uncertainty

allocating degrees of freedom to each uncertainty component using the Welch-Satterthwaite equation

reporting results and uncertainties in the required formats.

Must provide evidence that demonstrates knowledge of:

the steps in the measurement, test or calibration involved

the difference between errors, corrections and uncertainties

uncertainty in the uncertainty estimation process

uncertainty components that are common to the use of an instrument

uncertainty components that arise due to the instrument being used under different conditions to those when it was calibrated

uncertainty components, such as:

calibration uncertainty

instability or drift in the calibrated instrument

repeatability of the results

resolution or readability of the instrument

environmental influences, such as temperature, air pressure, humidity, vibration, electrical noise and gravity

reference material uncertainty

factors arising from using an instrument under a different operating environment or procedures (e.g. orientation of a transducer and immersion depth of a temperature probe)

reproducibility of quality control data

procedures for determining the uncertainty components associated with each of the inputs and whether they are significant and for applying appropriate corrections

manufacturer's specifications (e.g. instrument drift specification and reference materials)

procedures for determining uncertainty components from quality control data

the concept of degrees of freedom

the characteristics of a valid measurement

reporting requirements, such as the uncertainty in measurement (GUM), National Association of Testing Authorities (NATA) or other applicable reference material

confidence levels required by the National Measurement Act 1960 (e.g. 95% for most applications, but others may require a higher confidence level).

Judgement of competence must be based on holistic assessment of the evidence. Assessment methods must confirm consistency of performance over time, rather than a single assessment event.

This unit of competency is to be assessed in the workplace or a simulated workplace environment. A simulated workplace environment must reflect realistic operational workplace conditions that cover all aspects of workplace performance, including the environment, task skills, task management skills, contingency management skills and job role environment skills.

Foundation skills are integral to competent performance of the unit and should not be assessed separately.

Assessment processes and techniques must be appropriate to the language, literacy and numeracy requirements of the work being performed and the needs of the candidate.

Knowledge evidence may be collected concurrently with performance evidence or through an independent process, such as workbooks, written assessments or interviews (provided a record is kept in each case).

This unit of competency may be assessed with:

MSL904001 Perform standard calibrations

MSL905001 Perform non-standard calibrations

Holistic assessment methods include:

review of data worksheets, calculations, spreadsheets, databases, statistical analysis, graphs and/or tables prepared by the candidate

questions to assess understanding of relevant procedures, trends in data and sources of uncertainty

review of reports and records prepared by the candidate

feedback from supervisors and peers regarding the candidate's ability to estimate uncertainty in accordance with workplace procedures.

Access is required to all instruments, equipment, materials, workplace documentation, procedures and specifications associated with this unit, including, but not limited to:

data sets and records

test methods and description of test set-up

computer and relevant software or laboratory information system

workplace procedures.

Assessors must satisfy the assessor competency requirements that are in place at the time of the assessment as set by the VET regulator.

The assessor must demonstrate both technical competency and currency.

Technical competence can be demonstrated through:

relevant VET or other qualification/Statement of Attainment AND/OR

relevant workplace experience.

Currency can be demonstrated through:

performing the competency being assessed as part of current employment OR

having consulted with a laboratory about performing the competency being assessed within the last twelve months.

Copy and paste from the following performance criteria to create an observation checklist for each task. When you have finished writing your assessment tool every one of these must have been addressed, preferably several times in a variety of contexts. To ensure this occurs download the assessment matrix for the unit; enter each assessment task as a column header and place check marks against each performance criteria that task addresses.

Observation Checklist

Tasks to be observed according to workplace/college/TAFE policy and procedures, relevant legislation and Codes of Practice Yes No Comments/feedback
Specify an equation for the measurement 
List uncertainty components that are associated with each input in the equation 
Calculate the standard deviations and standard deviation of the mean from the measurement results 
Use calibration reports, manufacturer's specifications, quality control and validation data, and experimental data to collect other available information on the uncertainty components 
Allocate an appropriate distribution for each uncertainty component 
Calculate the standard uncertainties 
Calculate the sensitivity coefficient for each uncertainty component 
Calculate a combined standard uncertainty 
Determine an appropriate coverage factor based on the degrees of freedom associated with each uncertainty component 
Calculate the expanded uncertainty 
Report the result and uncertainty to an appropriate number of significant figures 
Report the confidence level and coverage factor 
Determine the appropriateness of the size of the expanded uncertainty relative to the tolerance or required accuracy of the test 
Determine the fitness for purpose of the expanded uncertainty relative to the use of the measurement result 

Forms

Assessment Cover Sheet

MSL925002 - Analyse measurements and estimate uncertainties
Assessment task 1: [title]

Student name:

Student ID:

I declare that the assessment tasks submitted for this unit are my own work.

Student signature:

Result: Competent Not yet competent

Feedback to student

 

 

 

 

 

 

 

 

Assessor name:

Signature:

Date:


Assessment Record Sheet

MSL925002 - Analyse measurements and estimate uncertainties

Student name:

Student ID:

Assessment task 1: [title] Result: Competent Not yet competent

(add lines for each task)

Feedback to student:

 

 

 

 

 

 

 

 

Overall assessment result: Competent Not yet competent

Assessor name:

Signature:

Date:

Student signature:

Date: